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5 CROSS REFERENCE TO RELATED APPLICATIONS 

This application claims the benefit of U.S. Provisional Application No. 
60/222,023, filed July 31, 2000, the disclosure of which, and subsequently filed 
applications converted therefrom, is herein incorporated by reference. 

10 BACKGROUND OF THE INVENTION 

The present invention relates generally to methods and apparatuses for obtaining 

information about and/or purchasing pre-recorded music, and more particularly to a 

method and system for obtaining information about and/or purchasing pre-recorded 

music while listening to the music at any location. 
1 5 When listening to music, people often want to identify a song currently being 

played on an audio system, such as a radio, but can identify neither the title nor the artist. 

The listener may simply be interested in the artist, title, lyrics, genre, or other information 
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about the music. The listener may also be interested in obtaining a copy of the music, i.e., 
purchasing the music. 

Frequently, the radio station announcer does not state the title, recording artist or 
other information about the song at the moment when the listener is interested in this 

5 information. Even if this information was announced, it may have been announced before 
the song was played and at that time the listener was not interested or was not then tuned 
to the station. The listener must then wait until hearing the song again and hope that the 
title and artist are announced at that time. 

Even when information about the song is announced and heard, there are 

1 0 situations in which such information cannot easily be retained, such as when a listener is 
operating in a noisy environment, such as in an automobile, or when a listener does not 
have pen and paper at the ready. This situation presents problems to radio listeners and 
potential music purchasers and sellers alike. Market studies have shown that many radio 
listeners prefer radio stations that always announce the name of every song and artist. 

1 5 A significant number of radio listeners seeking to buy music at record stores are 

often unable to remember the name of a song or the recording artist by the time they enter 
a music store to purchase the music. In fact, the sheer number of music recordings 
available for purchase in a music store can be so imposing that many novice music 
purchasers do not venture into such stores to purchase music, despite wishing to purchase 

20 music. Music fans would buy more music if they had immediate information about the 
title of the song and artist as it is being played, such as on the radio or other location 
remote from typical retailing locations. 
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Methods exist for automatically identifying music from a high-quality, relatively 
lengthy recording. For example, companies that monitor radio broadcasts to determine 
copyright and publishing royalties and to construct listings of current best-selling or most 
popular recording i.e., "top charts", employ certain techniques for identifying copyrighted 

5 songs from the broadcast. However, these methods require a high quality piece or excerpt 
of the song (referred to as a "sample") and are ineffective on short noisy samples. 
Usually, these methods require a clear signal that is a direct, high-quality connection to 
the radio output before it is broadcast to prevent noise from corrupting the sample. 
Consequently, these methods cannot work in a noisy environment using short samples. 

1 0 Many times unidentified music is heard when riding in a car (or at another 

similarly inconvenient location). Moreover, when a listener decides he wishes to know 
the identity of a particular song being played, it is usually well into the song. Therefore, 
even if the listener were to begin recording the song at the moment he decides he wishes 
to know the identity of the song, the sample would be relatively short and possibly noisy 

1 5 depending upon the quality of the audio recording and the recording environment. 

Certainly, most listeners do not carry high quality recording equipment with them when 
traveling in a car. 

Moreover, even if the listener knows the identity of a song, as time passes the 
desire to obtain a copy of the song also passes. This is the so-called impulse purchase 
20 phenomenon, which is well known to retailers. The impulse purchase phenomenon is 

particularly strong where the listener has not heard the song before, and thus is unfamiliar 
with the title and/or recording artist. Unfortunately, there is currently no way for a music 
seller to take advantage of a potential impulse purchase resulting from a listener hearing a 
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song (for perhaps the first time) in a car or other location that is remote from normal 
retail locations. 

The present invention is therefore also directed to the problem of developing a 
method and system for both identifying music and/or enabling music retailers to take 
advantage of impulse purchase phenomena. 

SUMMARY OF THE INVENTION 

The present invention solves these and other problems through a method and 
system for providing a user with an ability to interactively engage with a service to 
trigger one or more of a variety of experiences, transactions or events by capturing a 
sample of an audio stream to which the user is exposed, and delivering that sample to the 
service. Such experiences, transactions and events, include purchases by the user, 
delivery of information from the service to the user, the execution of tasks and 
instructions by the service on the user's behalf, or other interactions that are responsive to 
the user's wishes. 

Thus, according to one exemplary embodiment of the present invention, a service 
utilizes a system for identifying songs and music from a relatively short sample and 
enabling a user to interact with the service to immediately purchase a recording of the 
identified music remotely. In this embodiment, a user captures a relatively short sample 
of music being played over a music playing device such as a radio, by placing a call with 
a mobile telephone using a predetermined telephone number and playing the music into 
the handset of the telephone. A system at the other end of the telephone identifies the 
song to the user in real-time (i.e., within the duration of the short call). Employing an 
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interactive voice response unit ("IVR"), the service enables the user to immediately 
purchase a recording of the identified song. The present invention thus takes advantage of 
the impulse purchase phenomenon by providing a mechanism that guarantees that the 
user is at the height of interest in the particular music. Coupled with this mechanism is 
the capability of the user to purchase the music in a very short transaction to ensure the 
interest does not fade. 

Other exemplary embodiments of the invention provide for pre-authorized 
purchases of identified songs and the detection of unauthorized use of copyrighted 
materials. 



BRIEF DESCRIPTION OF THE DRAWINGS 

FIG 1 depicts an operational block diagram of an exemplary embodiment in 
accordance with one aspect of the present invention. 

FIG 2 depicts an operational block diagram of details of the signal identification 
1 5 block in FIG 1 in accordance with another aspect of the present invention. 

'TIG-3-depi^-an o^ of details of the reporting and 

transaction block in FIG 1 in accordance with yet another aspecfoTBT^ pi c5ent4*uientiorL 

FIG 4 depicts an operational block diagram of an interaction between a user and 
interactive voice response unit in accordance with yet another aspect of the present 
20 invention. 
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OFT AILED DESCRIPTION 

At this point, it is worthy to note that any reference herein to "one embodiment" 
or "an embodiment" means that a particular feature, structure, or characteristic described 
in connection with the embodiment is included in at least one embodiment of the 

5 invention. The appearances of the phrase "in one embodiment" in various places herein 
are not necessarily all referring to the same embodiment. 

The present invention includes a method and system for providing a user with an 
ability to interactively engage with a service to trigger a variety of experiences, 
transactions, or events by capturing a sample of an audio stream to which the user is 

1 0 exposed and delivering that sample to the service. It is noted that the terms "service" and 
"service provider" and "system" as used herein include a service, service provider, and 
system, respectively, which employ the various principles of the present invention unless 
otherwise indicated. 

The audio stream can take any form where a message or information (i.e., 

1 5 content) is available for the user to experience, and may come from many sources, radio 
and television, pre-recorded audio, signals on internet and computer-based systems, 
telephones, and even live demonstrations or performances. Using a sampling device, such 
as an ordinary mobile (or cellular) phone, the user captures a sample of the audio stream 
and transmits the sample to a service provider employing the present invention. 

20 The service provider may employ the sample by itself, may derive information 

from the sample, may use data known about the user (e.g., the user's identity and/or user 
profile), may prompt the user for additional information, or may employ a combination of 
all such inputs, to trigger an experience, transaction or event that is responsive to the 
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user's needs. As described in more detail in the various embodiments of the invention 
below, such experiences, transactions and events include purchases of merchandise or 
services by the user, delivery of information from the service to the user, the execution of 
tasks and instructions by the service on the user's behalf, and other interactions. 

5 An exemplary embodiment of the present invention provides a method and 

system for both identifying songs and music from a sample captured and transmitted 
using a mobile telephone, and providing a listener with an immediate opportunity to 
purchase the identified music. As mobile telephones are becoming more prevalent in the 
marketplace, listeners are likely to have access to a mobile telephone when hearing a 

1 0 song they would like to identify. 

When desiring to obtain the identity or other information about a song being 
played, the user (i.e., listener) dials a predetermined telephone number using a mobile 
telephone (or any other available telephone, including landline, cordless, etc.), and then 
holds the telephone handset within audible distance of the source of the audio signal. The 

1 5 telephone number may be programmed into the phone by the user, or the number may be 
preprogrammed into the phone before its sold in some applications of the invention. 

A system at the other end of the telephone automatically answers the phone, 
compares the sample to those music recordings in its database and then identifies the 
song to the user. Using an IVR (or a live operator and/or a combination of IVR and live 

20 operator; all alternatives collectively referred to below as "IVR"), the user is then 

provided the opportunity to immediately place an order for a recording of the identified 
music, for example, an album on CD or tape that contains the selected track. 
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The purchase may be physically fulfilled (i.e., forwarded to the user) in any 
number of different ways, such as regular mail, express mail, overnight delivery, courier, 
etc. Alternatively, digital delivery and fulfillment techniques may also be used, including 
digital downloads to users' PCs, wireless internet devices, hand held personal computers, 

5 palm pilots, or mobile telephones. 

Other purchase options may also be provided to the user in accordance with other 
aspect of the present invention. In addition to being able to purchase the identified music, 
the user may purchase other merchandise (which could be related to the music or not). 
For example, after purchasing the album containing the identified song, the user may 

10 choose to purchase tickets to an upcoming performance by the song's artist or sending 
flowers to a loved one who shares that "special song". 

In another exemplary embodiment of the invention, the user can set up an 
account with the service where the user is enabled to select in advance to automatically 
purchase an identified music selection by simply calling a predetermined number. That is, 

1 5 the service treats a call to a dedicated number as an order for the identified music. 
According to this embodiment, when the user calls the predetermined number, the 
computer answers the telephone, identifies the user by the phone's ID, and, after the song 
selection is identified, a sales order is automatically placed. The user's account is then 
debited, and the selection is forwarded to the user. The selection could even be added to a 

20 compilation being created for the user from other similarly identified selections. 

For example, the system could keep adding identified songs to a personal 
compilation until the compilation reaches the limit of the recording media. Once 
reaching the limit of the recording media, such as a compact disk, the system would issue 
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the media to the user. This system enables recording artists and record companies to 
receive the proper royalties for their music while enabling listeners to create individual 
music compilations on any desired media. 

On some embodiments of the invention, the service relies upon a profile for the 

5 user that is created in advance of the call (but may be updated during a call as described 
below). A "user profile" may typically include general information about the user such as 
name, address, preferred method of payment (i.e., credit card pre-authorization), and set 
dollar limits on purchases. In addition, service-specific information regarding the user 
may also be included in the profile, such as demographic and user-identified preference 

1 0 information, to facilitate the service tailoring the transaction to fit a particular user. 

Moreover, with automatic telephone number identification, i.e., "caller ID", profiles can 
be built without prior user registration. 

Age, education, residence, gender, occupation, and personal interests, likes and 
dislikes, among other criteria, may be employed to most effectively match transaction 

15 offers to users' interests and purchase habits. For example, one particular customer of the 
service may have a user profile that indicates that the user is a member of a demographic 
group that is music-sawy and aware of music trends. After offering to sell a recording of 
the song selected by the user, the service could offer to sell a recording by an artist that is 
"moving up the charts" in popularity. Thus, by employing a user profile in some 

20 applications of the invention, a higher transaction closing rate may be realized as offers 
are more accurately targeted to users who may be predisposed to view the offer 
favorably. In one embodiment of this aspect of the invention, the system could play a 
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sample of the other recording over the telephone as part of the transaction, thereby 
ensuring that the user was interested in this recording. 

It is noted that user profiles are often dynamic and that the present invention 
includes an ability to update and change a user profile in response to changes in usage 
5 patterns (as described below), market and industry trends, user input, and other factors. 

A feature of the invention related to user profile is usage tracking. By tracking the 
frequency and time of access, the type of music sought to be identified, and purchase 
history, for example, of a user, the service can gain additional insight into factors which 
may influence a user. Patterns of usage may be derived which may allow predictive 
10 modeling to be utilized, for example, to enhance and refine service offerings. The system 
of the present invention can thus differentiate between repeat users (who heavily access 
the service) and casual users (who occasionally or infrequently use the service) and adjust 
the options, offers, and interactive scripting (as described below) so that interest and 
enthusiasm is maintained among the service's customers. It is contemplated that the user 
15 profile and usage tracking/pattern features described here may be used in other 
embodiments and applications of the inventions as well. 

There is an understanding of the fact that there may be some instances where 
recognition of an unidentified sound or song is not possible. In some cases this may be 
due to very short or noisy samples that challenge even the very capable technology 
20 platform of the present invention, or more likely, there is not an entry in the service's 
database that matches the sample even under the best conditions. 

While mass storage of a vast library of recorded music and sounds is technically 
feasible, clearly it is not economical for a service provider to store data corresponding to 
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every song or sound that exists in the world. Moreover, as barriers to publishing come 
down (through increased use of such distribution channels as the internet), it is expected 
that the rate at which new music is introduced to the public will continue to increase. As a 
result, it is expected that the entries to the service's database will always lag the newest 
5 (or most obscure) releases. 

The present invention contemplates that such unidentifiable samples may be 
advantageously utilized. Users are afforded the opportunity to "stump" the service, 
acquaintances, or other users. As evidenced by the popularity of radio programming 
where the disk jockey plays a small snippet of a song and then rewards the listener who 

10 correctly identifies the "mystery" song, many users of the present service are expected to 
embrace this enjoyable aspect of the invention. Accordingly, in some applications of the 
invention users may store, retrieve, and forward the captured samples (for example, to 
other users). A service provider can thus implement contests, games, and promotions 
using the present invention to implement a modern day "Name that Tune." Such activities 

1 5 may be expected to heighten interest in the service as a whole. 

In another aspect of the invention, in addition to having the option of purchasing 
the identified song, the user may initiate other types of interactive experiences, events, or 
transactions with the system during a call. Such interactive experiences, events, and 
transactions may include purchases of merchandise or services, the receipt of 

20 information, the storage and manipulation of archived data, and an ability to command 
the system to perform desired tasks or instructions. Thus, this aspect of the invention 
provides the user with a common and consistent interface with the service to perform 
multiple interactive transactions. 
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For example, through the interactive voice response unit, the user may select from 
a number of options. An interactive voice response unit advantageously provides accurate 
and fast interactions between the user and the service and minimizes the amount of 
intervention by service personnel. Thus, playing a programmed audio script to the user, 
and receiving DTMF (dual tone multi-frequency or "touch") tones or voice commands 
from the user, the interactive voice response unit interacts with the user by providing the 
option to hear a clip of the identified recorded song or hear clips from other songs (e.g., 
tracks on the same album or from the same artist as the identified song, or music that is 
determined by the service to be of interest to the caller by a usage pattern or user profile). 

The user may be given the option of obtaining more information about the song, 
artist, or album. Examples of such information include song genre (e.g., folk, rock, R&B, 
country, rap, etc.) song lyrics, trivia, liner notes, production notes, instrumentation, 
musician identification, song position in music ranking services, such as Billboard®, and 
calendar of live appearances by the artist. Service-specific information may also 
furnished such as the tracks that the service identifies with the greatest frequency to the 
service subscribers on a given day. For some users, a desire to gain such information may 
be equal or stronger than the desire to know a song's title and artist. 

Data archiving and manipulation may be implemented as another interactive user 
option. This aspect of the invention provides for the user to save the identification and/or 
other information on a local storage medium for future access. And, as noted above, users 
may wish to store the captured samples themselves. A typical voice mailbox system may 
be utilized to implement such functionality, or alternatively, a web-based interface may 
be established with user-allocated disk space on a local server. Such access may be 
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gained using the same telephone interface used with song identification, or an alternative 
platform (i.e., one that is accessed by dialing a different number). 

Alternatively, saved data may be accessed and/or manipulated using such 
interfaces as a web browser with internet access. Such data archiving and manipulation 
feature of the invention may be used to implement service features that would, for 
example, allow users to store information on identified tracks and play them back on 
demand, or allow others to access the stored tracks. 

Additionally, users may create their own music compilations and share them with 
others, using for example, MP3 storage and access protocols. Other information input 
from a user may also be stored, for example, to annotate a track or compilation with the 
user's thoughts and comments. Captured samples may be stored and manipulated as well 
to facilitate contests and games (as described above) or to implement other functions that 
may require accessing the "raw" sample. 

Another aspect of the user and service interaction provided by the present 
invention includes an ability of the service to implement tasks and instructions directed 
by the user. One notable example of this feature includes the connection of a live operator 
(i.e., customer service personnel) to the user upon command. This feature additionally 
allows a user to interact with others users, non-users, or potential users of the service, or a 
user may interact with other systems. 

For example, the user may instruct the service to send a recommendation for the 
identified song to a friend via email or SMS or a clip of the song to the friend's mobile 
telephone or service-sponsored storage site (e.g., voice mailbox or allocated disk space). 
The user could also instruct the service to ship a recording by a featured artist, or other 
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merchandise (both music-related and non-music-related merchandise), as a gift. Games, 
contests, and promotional events involving interactive user participation may also be 
readily implemented using this inventive feature. 

In addition, the service, acting as a portal or gateway or referring agent, may also 
5 provide the user with access to other systems, information, and services hosted or 
provided by third parties. 

Such interactive experiences may include the delivery of advertising or 
promotional messages to the user from the system. Such advertising may be general or 
targeted to a user based on a user profile. A user may also be directed (for example, via 

□ 10 call forwarding, or via web link) to fulfillment partners of the service provider to either 
fulfill a purchase request or user-driven instruction in instances where such partners are 
used. Alternatively, direct links to fulfillment partners, advertisers, and other systems 

j t i may be provided as an option to a user in themselves. 

□ In addition to the interaction that is implemented between the user and system 

cx 
.f 5 
e» 

" [ i 1 5 during a call, the user may also be directed to other sources of information that are 
;J external to the system, for example, internet websites. Such websites may host content 

that is specifically related to the identified song, other music, or other topics of more 
general interest. The website may be configured to provide similar or additional 
interactive experiences to the user. 
20 . In another exemplary embodiment of the invention, a user experiencing a 

multimedia presentation, including an audio track, may capture and transmit a sample of 
content in the show to the service to indicate an interest in a product or service that may 
be featured. 
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For example, a user may capture an audio sample of a commercial advertisement 
for a particular product or service from a broadcast TV presentation. Upon receipt and 
identification of the product or service from the captured sample, the service may provide 
a purchase option to the user, furnish more detailed product or service information, or 
other provide other optional interactions as described above. 

Fulfillment of the desired transaction to the user can be accomplished by the 
service directly, or through fulfillment partners as appropriate. This embodiment may be 
particularly beneficial when the advertiser does not include ordering or retail location 
information in the content of the advertisement. For example, a local retailer may not be 
known to the advertiser or it may be impractical for the advertiser to identify such 
retailers during the course of the advertisement (as in cases where the advertisement is 
distributed over a broad geographic region). Or, the advertiser may not equipped with all 
the necessary facilities (such as an IVR) to receive direct orders from consumers. 

Thus, this embodiment of the invention provides a benefit to both user and 
advertiser alike. For the user, the service implements a desired transaction that would be 
more difficult and less convenient if the user employed conventional means to achieve 
the same end. And, by locating and identifying users who are ready to purchase to 
advertisers, the service can increase product deliveries by acting as a referral agent (by 
establishing a path of communication between the user/consumer and advertiser), or by 
acting as a "virtual" IVR for the advertiser (where the service provides the customer 
service interface between the user/consumer and the advertiser, and the advertiser acts as 
a defacto fulfillment partner of the service). 
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The principles of this embodiment may also be readily applied to paid television 
programming (i.e., "infomercials") and home shopping broadcasts. In the case of paid 
television programming, product information segments are typically interspersed with 
segments that provide ordering information (during such segments, a viewer is generally 
provided product pricing information and directed to a toll-free telephone number to 
place an order with a live customer service representative). The content recognition 
technology employed by the present invention allows for faster, more accurate, purchases 
on impulse (even at times during the presentation when the toll-free number is not 
displayed) and provides an alternative ordering path to the 800 IVR. 

In addition, in cases where higher valued products are featured that normally 
require more deliberation before purchase decisions are made, users may desire more 
information to assist in their decision making but often be reluctant to interact with a live 
representative to obtain it (i.e., the user is not ready to buy and wishes to avoid being 
pressured into buying). By contrast, user/consumers employing the present common user 
interface feature would be comfortable in knowing that such additional information could 
be readily obtained without human contact as the interface is seamless and consistent 
across all experiences, events and transactions in accordance with the invention. 

In the case of home shopping broadcasts, ordering information is generally 
displayed at all times during the broadcast. Consumers typically place orders via toll-free 
IVRs by identifying item numbers, color, size and other identification information for the 
products desired for purchase to the home shopping service. While such methods are 
satisfactory, use of the present invention can enhance the home shopping experience for 
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many users through its interactive interface that facilitates the exchange of ordering 
information using the common user interface feature described above. 

In another exemplary embodiment of the present invention, the unauthorized use 
of copyrighted material may be detected. Recognition of copyrighted material without 
embedded watermarks is also provided. This aspect of the present invention enables an 
operator of a server to sample music recordings being transmitted over its server to 
ascertain whether the users are violating copyright laws. By identifying copyright 
violations at the server level, a web site operator can potentially avoid being accused of 
facilitating copyright infringement. 

Other techniques for obtaining the music sample may be employed due to the 
ability of the algorithm utilized in the present invention to process a relatively short 
sample and generate a positive identification in the presence of relatively large amounts 
of noise (both background noise in the user's environment and noise resulting from signal 
compression and/or impairments along the signal transmission path). 

For example, the music sample can be captured and sent to the system using many 
real-time sampling techniques. Thus, in various embodiments of the invention, a user 
may capture and transmit a sample using, for example, a standard (i.e., wired) telephone, 
internet-streaming socket, or voice-over-IP transceiver. In other embodiments, a sample 
may be first recorded and then later transmitted to the system for identification, using for 
example, tape recording, CD writing, answering machine, digital voice recorder, 
uploaded WAV file, and the like. 

Thus, users of the present inventive system can employ telephony or data 
communications devices including ordinary or mobile telephones, PCs, internet access 
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devices, wireless internet devices, personal data assistants ("PDAs"), wireless 
information devices, two-way pagers, and other devices that are equipped with an ability 
to capture a media sample in accordance with the principles of the present invention. 
The present invention provides tremendous advantages to users to facilitate the 

5 identification of songs anywhere and anytime. The identification occurs on a real-time 
basis, that is, the user can expect the information to be provided within the duration of a 
short transaction with (the service such as a one minute telephone call). The user does not 
need to wait to receive the desired information at a later time. The user simply and easily 
accesses an interactive voice response unit, dial up data interface or website, allows the 

1 0 microphone on the phone or PC to capture the music being played or performed 

anywhere for a few seconds, and, within seconds, learns the name of the song and the 
artist, as well as a range of additional information upon request. Users can then instantly 
purchase the music (in multiple formats), or save the information and access it later on a 
personalized web site (which may be configured, for example, to provide "personalized 

1 5 radio " i.e., content aggregation according to the user's taste, using streaming audio), or 
perform several other actions from their telephone, internet access device, personal 
digital assistant (PDA), or PC. 

By providing for real-time identification of songs being played or performed 
anywhere, the present invention solves the problems identified above and provides 

20 significant advantages for both buyers and sellers of music. First, music buyers 

employing the present invention are provided with the song and artist information that 
they wish to know, as well as the ability to purchase a song or an entire album as soon as 
they hear it, i.e., at the peak of their interest in the song. Second, sellers of music, 
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including record companies and retailers, will benefit from heightened impulse 
purchases, as customers are able to recognize and remember songs they hear. 

Furthermore, the present invention facilitates the creation of a rich customer 
information database that may be used to fine tune marketing campaigns by radio station 
and record companies and retailers, such as promotions and targeted marketing. Such 
data may be useful for the music industry to learn valuable new insights on customers' 
music preferences. The reactions of the users of this service to new music will provide 
valuable predictive information to recording labels and musicians attempting to market 
their recordings. Thus, the present invention provides for the first time a direct 
connection to the heartbeat of the music consumers for music retailers, record companies 
and radio stations. 

The present invention provides a service operator with a multitude of revenue 
opportunities. The immediate purchase opportunity for the user afforded by the invention 
could allow music sellers to provide monetary and other incentives to the service 
provider so that the identification service is offered free of charge or at subsidized rate 
(i.e., a rate that may be less than the cost of providing the service). Alternatively, the 
service could obtain revenue from the price of the telephone call, e.g., similar to 900 toll 
call telephone service, in which the user pays a toll set by the 900 number system 
operator above and beyond the normal toll fee for placing the call. 

The service may also share in the increased revenue realized by telephone 
companies and internet service providers ("ISPs") as users increase their usage of access 
facilities to use the service. In addition, subscription arrangements with users may be 
established and revenue received for the service's delivery of advertising and 
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promotional materials to the user. Revenue may also be realized through merchandise 
retailing and fees collected from fulfillment partners. Revenue may also be realized by 
selling information or advertising to companies by leveraging the user-specific 
knowledge acquired through use of the service. 

The present invention employs an audio pattern-recognition technology that can 
be used to recognize a sound signal within a short noisy and distorted audio sample, 
which technology includes a robust recognition engine that is context free, convenient, 
fast, and noise immune. The audio pattern recognition technology is robust in that it 
works with highly distorted and noisy signals. In contrast, current music recognition 
engines require high-quality signals. The audio pattern technology is context free in that 
it does not rely on the knowledge of the source of the broadcasting. Prerecorded sounds 
can be recognized in circumstances where the source is unknown (for examples in bars 
and restaurants, on the street, in cars, airplanes, public transportation, etc.). The audio 
pattern technology is convenient in that excerpts of only a few seconds from any unique 
portion of the entire song rendition are sufficient to enable recognition. Other techniques 
require comparatively long samples of the music to positively identify it. The audio 
pattern technology of the present invention is fast because it uses an algorithm that can 
search a very large database in a very short period of time. 

The present invention overcomes the shortcomings suffered by various other 
known arrangements that attempt to solve the song identification problems mentioned 
above. One such arrangement identifies information (i.e., songs and commercials) 
broadcast by certain major radio systems by relying on a third-party identification service 
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and the manual input by the user of data specifying the source of the broadcast 
information. 

Another known arrangement requires users to carry a keychain, which can 
timestamp the moment a song is played on the radio, and later enable purchases when the 
device is synchronized with an internet-connected computer. Users of such arrangements 
suffer an undesirable delay before learning the identity of the song, thereby diminishing 
the power of the impulse purchase phenomenon. In addition to the delay limitations, both 
above-described arrangements can only identify songs aired on radio stations. 

Other music recognition arrangements have avoided attempting to perform music 
identification directly. Instead, they rely on contextual information, such as a timestamp 
plus radio station in order to look up a playlist provided by a third-party source. Such 
third-party information must be gathered using playlists submitted by radio stations, 
causing dependencies on business relationships, reducing radio station participation, or 
problems when the Disc Jockey changes the actual song played at the last minute. Users 
of certain such arrangements are limited to only the radio stations with which they have 
registered. Alternatively, third-party playlist information can also be provided by a music 
tracking services, but such services usually only track radio broadcasts for the largest 
radio stations in major metropolitan areas. 

Another limitation with these known music recognition arrangements is that they 
cannot be used in the conditions under which present invention operates with the same 
effectiveness. Current recognition technologies do not lend themselves to robust 
recognition as they fail under less than ideal conditions such as those encountered due to 
noise, dropouts, interference, bandlimiting, and voice-quality digital compression. 
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Signals that are transmitted through additional mediums that are subject to linear and 
nonlinear distortion cause search methods that rely on cross-correlation or statistical 
moments to fail. Additionally, the arrangement of the present invention can recognize the 
music with as little as 5 to 15 seconds of sampling time, depending on the signal quality 
5 and size of the database, potentially allowing recognition and purchasing transactions to 
be carried out within a one-minute telephone call. 

Other known music recognition arrangements embed a perceptually inaudible 
watermark or other side-channel information to identify the music. To take advantage of 
such embedded information the user must have a special decoding device, perhaps built 

1 0 into the receiver, to obtain the information. Using a telephone, or a computer with a web 
browser, it is possible to obtain play information from the radio station. But this requires 
the user to know which station he or she is listening to, as well as how to contact the 
station. By comparison, the method and system of the present invention is context-free, 
requiring only that the user submit the sampled music information directly for 

15 identification. No watermarks, side information, special devices, or knowledge of radio 
station contact information are necessary. 
£^\(V/ "R^n-ing now to FIG 1, there is shown an operational block diagram of an 



exemplary embodiments 00 according to one aspect of the present invention. A signal 



source 101 in the environmentgfcnQrates a signal 103 that is captured by signal capture 
20 block 105 and transmitted to signal identification block 1 10 on line 107. Signal 

identification block 1 10 is described in greater detail beltr^Signal identification block 
1 10 identifies the signal 103 from signal source 101 and passes the identification 
information (i.e., song ID) to reporting and transaction block 1 15 via line 1 13. lfe$r 
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ident ification btedU^Oadditionally provides the identity of the user (not shown) to the 
reporting and transaction block 115 on lmTTT^r^her^ the user may be informed as to 
the identity of the signal as described more fully in the text accompaaj4iigJFTG 3 below. 

In some applications of the invention, as discussed further below, it may be 
preferable to have user identification block 120 operationally coupled to, or alternatively 
incorporate, a user database where user data (and data related to other user options), 
among other features, may be read from and written to (not shown in FIG 1). The user 
may optionally carry out transactions regarding this signal identity as described in greater 
detail below and in Appendix 1. Billing data from user billing block 125 on line 127 may 
be used to generate a bill to the user commensurate with the extent of the usage of the 
signal identification service aspect of the invention as described herein. Additionally, 
statistics regarding the signal identification, signal identified, and/or the user's usage of 
the service may be logged by statistical data logger 130. Such data is of interest in market 
research, and has economic value. In such embodiments of the invention, user 
identification information is passed from user identification block 120 to statistical data 
logger 130 on line 124 as shown in FIG 1 and reporting and transaction data from 
reporting and transaction block 1 15 is passed via line 129. In some applications of the 
invention it may be desirable for statistical data logged by logger 130 to be fed to user 
identification block 120, via line 124, for example to facilitate the update of user 
identification information. 

The signal 103 generated by signal source 101 in FIG 1 may be any kind of signal 
in the environment that is a rendition of a signal indexed in a database within the signal 
identification block 110. Examples of signal 103 include recorded music, radio broadcast 
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programs, advertisements, and other such signals of interest. Accordingly, the signal may 
be in the form of acoustical waves, radio waves, digital audio PCM stream, compressed 
digital audio stream (such as Dolby Digital or MP3), internet streaming broadcast, or any 
other such manner of transmitting such pre-recorded material. Furthermore, signal 103 
may be degraded as a result of background noise (such as that encountered while in a 
moving car), talking voices, transmission errors and impairments, interference, time 
warping, compression, quantization, filtering, or other such distortions of the original. 

Signal capture block 105 captures a sample of signal 103 and provides it in a 
format suitable for processing by the signal identification block 110. Illustrative 
embodiments of signal capture 105 devices include, but are not limited to, microphone, 
telephone, mobile telephone, tape recorder, digital voice recorder, answering machine, 
radio receiver, walkie-talkie, internet streaming socket, voice-over-IP transceiver, or 
other such signal capturing device. Typically then, the signal capture device is 
incorporated into a device that the user employs in a location remote from the service. 
Conventional devices like mobile and regular telephones, PCs, radios, and other 
recording and communication devices that users already own or use everyday for other 
purposes may conveniently be used, without modification, to practice the present 
invention. Upon capture of a sample of signal 103, signal capture block 105 delivers the 
sample to the system via line 107, as indicated. 

User identification block 120 identifies the user to the song recognition 
arrangement of the present invention, and may optionally be operationally coupled to the 
signal identification block 1 10 via line 122. Examples of devices which generate the 
appropriate identification for use with user identification block 120 may include caller ID 
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on a POTS (Plain Old Telephone Service) line or a mobile telephone, internet IP address 
of a terminal sending in the captured signal, or a cookie file stored on an internet browser 
on the user's terminal. In such implementations, the user's signal capture block 105 
stores a user ID and identifies the user to the arrangement 100. In another illustrative 
example of user identification block 120, the user may be required to enter an account 
code, for example by keying it in on a touchtone pad on a telephone or saying a pass 
phrase while signing on to a service incorporating the principles of the present invention 
if dialing in. Alternatively, the user may be identified by inserting an object carrying 
identification codes into a terminal. Examples of this include a credit card, ATM card, or 
Dallas Semiconductor Java Ring. The user may also be identified by a biometric device 
to scan fingerprints, retinae, palm print, or other such physical characteristics of the user. 
A speaker identification system to identify the user by vocal characteristics is another 
alternative method. User identification block 120 is an optional component of the present 
arrangement, which is employed if billing and tracking of user activity is desired. 

As shown in FIG 4, the elements shown and described in FIG 1 are typically 
associated with entities that are independent of one another. Signal source 101 is typically 
associated with a media operator or content provider such as radio or television 
broadcasters, CATV provider, internet service providers, private network or LAN 
operators, and the like. It is important to emphasize that the present invention 
contemplates that the signal source may comprise a live demonstration or performance, 
taking place for example, at a nightclub, bar, or discotheque. Signal capture block 105 is 
associated with users, however, such association may be merely temporary, as public 
access devices (e.g., public telephones and internet access facilities) may be readily used, 
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without modification, in order to realize the benefits provided by the present invention. 
Signal capture block 105 represents features and functionalities that, for example, are 
implemented by the microphone and associated transceiver circuits in a user's mobile 
telephone. 

5 The remaining elements of FIG 1, are collectively associated as indicated in FIG 4 

with a service provider. Signal identification block 110, user identification block 120, 
reporting and transaction block 115, user billing block 125 and statistical data logger 130 
represent features and functionalities of an integrated system that form key elements of a 
IVR arrangement that may be particularly useful in some applications of the invention. In 

0 1 0 such IVR arrangements, these collected elements are typically implemented in a system 
!; ? : formed by one or more CPUs. The IVR is identified by reference numeral 450 in FIG 4. 
j fi As depicted in FIG 4, a media provider 410 utilizes signal source 101 which 

\*& transmits signal 103 via media distribution network 420 which may be arranged from 

□ network and transmission elements or other channelized distribution arrangements (as for 

sz. 

1 ~ 15 example, with copper or fiber networks for data or telephony services) or a free 

'Si 

space/transmitter infrastructure array (as for example with radio and television 
broadcasts, satellite systems, and cellular/PCS/GSM wireless telephone services or 
networks which operate according to short-range wireless protocols such as the 
Bluetooth™ wireless standard). Receiver/monitor 440 is employed by user 430 to receive 
20 signal 103 and transform the signal into a format that facilitates signal 103 to be 

monitored by the user. Receiver/monitor 440 may be a radio, television, PC, Hi-fi (i.e., 
stereo) with speakers or any other device that may be used to create a media experience 
(including audio and video) that may be monitored by the user. User 430 employing the 
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functionalities of signal capture block 105, for example using a mobile telephone, obtains 
a sample of signal 103 played on receiver/monitor 440, where the sample includes media 
content of interest selected by the user, such as a portion of an unidentified song. Thus, as 
shown in FIG 4, receiver/monitor 440 both outputs a rendition of signal 103 to signal 
capture block 105 and allows the user 430 to monitor signal 103. However, it is noted 
that signal capture block 105 may capture a sample of signal 103 via a direct connection 
to media distribution network 420 (i.e., not relying on receiver/monitor 440 or similar 
device for signal input). In such instances, the user monitoring of signal 103 is 
accomplished through other means or user monitoring is not performed. 

The captured sample of signal 103 is relayed by the user 430 to the IVR 450, as 
shown, via communication network 460 on line 107, as shown. Communication network 
460 may have a similar arrangement as distribution network 420, or may be unitary with 
distribution network 420. It should be noted, however, that in certain applications of the 
invention distribution network 420 may typically be characterized by unidirectional 
signal propagation (as in the case with broadcast radio and television or typical CATV 
headend systems) while communication network 460 may typically be characterized by 
bidirectional signal propagation (as in the case with the public switched telephone 
network and wireless or wired voice, data, and internet systems). Such bidirectional 
nature of communication network 460 is indicated by signal flow lines 107 and 108 as 
depicted in FIG 4. 

In accordance with the invention, the IVR 450 derives information or 
characteristics of sample of signal 103 including the identification of content contained 
therein (for example, the song ID). Such derived information may be returned to the user 
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430 from the IVR 450 using the same communication network 460 or other networks. 
The signal return path is indicated with lines 108 in FIG 4. And, as described above, the 
IVR may interact with the user and other entities. For illustrative purposes, such 
interaction pathways are depicted in FIG 4 as lines 489 and 482, input and output, 
5 respectively, via alternate network 480. Alternate network 480 may be a network of any 
type, however, in some applications of the invention it may be advantageous to employ 
private networks, dedicated lines, or other high-capacity transmission methods should 
high-bandwidth interactions be desired. Such bandwidth intensive interactions could 
occur, for example, between the service IVR and fulfillment partners such as record 
■«3 10 distributors. This communication path is shown in FIG 4 where fulfillment center 486 

'rbr 

interacts with IVR 450 using alternate network 480 and fulfills user purchase orders as 
I n indicated by line 488. Fulfillment center 486 may also interact with IVR 450 using 

communication network 460 over interaction pathways 492 and 494 as indicated. 
Q Thus, FIG 4 may serve to additionally highlight the principles applied in the 

j ^ 1 5 exemplary embodiment of the invention described in the Summary. A radio station 
!;i (media provider 410) employing signal source 101 broadcasts a song (signal 103) over 

the air (media distribution network 420), which is received on a radio (receiver/monitor 
440) of user 430. The song, which is unknown to user 430, is of interest. User 430 places 
a mobile telephone call over a wireless network (communication network 460) to the IVR 
20 450. The user 430 positions the microphone of his wireless telephone (signal capture 
device 105) to capture a sample of the music being played on the radio. The IVR 450 
receives the sample over the wireless network and derives the identity of the song. 
Optionally, the identity of the user may be derived from the user's mobile telephone 
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number that is sent from the communication network 460 to the IVR 450 (and user 
identification block 120, in particular) typically during call set up. The song identification 
is returned to back to the user's mobile telephone over the same network. Further 
interactions between the user 430 and the IVR 450 may occur, and if the user chooses to 
5 purchase a recording of the now-identified song, the IVR can send the purchase 

information to the service's distribution facility (fulfillment center 486) via data or voice 
communication using its local area PC network (alternate network 480) or via data or 
voice communication over communication network 460 as discussed above. The 
distribution center sends the selection to the user via overnight courier service (line 488). 

10 Similarly, alternate network 480 may be utilized to send an confirming email of the song 
identification and purchase, if applicable, to the user's email account as described in 
greater detail above. 

Signal identification block 1 10 (FIG 1) is described in detail in Appendix 1. In 
summary, and referring to FIG 2, the main modules in signal identification block 1 10 are 

1 5 the landmarking block 2 1 0, fingerprinting block 215, song database block 220, and 
alignment scan block 225. Landmarking block 210 receives the incoming captured 
sample signal via line 107, as described above, and computes sample landmark time 
points. This information is passed, as indicated in FIG 2, to fingerprinting block 215 via 
line 212, as well as the alignment scan block 225 on line 214. Fingerprinting block 215 

20 computes fingerprints from the captured sample signal at the landmarked time points, 
generating sample fingerprints. The sample fingerprints are output to song database 220 
on line 218 and are used to retrieve sets of matching song fingerprints stored in song 
database 220, these matching song fingerprints being associated with song landmark and 
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song ID values. The set of retrieved song ID and song landmark values are relayed to the 
alignment scan block 225, on lines 226 and 227, respectively, along with the associated 
sample landmark values on line 214 as indicated. Alignment scan block 225 sorts out sets 
of sample landmark and song landmark pairs, grouped by common song ID. Each set is 
5 scanned for linear correspondences in the pairs of landmarks and scored according to best 
fit. The song ID of the set with the highest score is the winning song ID which is output 
from alignment scan 225 on line 1 13. In an illustrative embodiment of the invention, the 
components forming signal identification block 1 10 are clustered together in a single 
computer system, such as an Intel-based PC or other workstation. In another illustrative 

10 embodiment of the invention, a networked cluster of CPUs may be used, with different 
software modules distributed among different processors in order to distribute the 
computational load. It may be preferable, in some applications of the invention, to use a 
cluster of Linux -based processors connected by a multi -processing bus architecture or a 
networking protocol such as the Beowulf cluster computing protocol, or a mixture of the 

1 5 two. In such an arrangement, song database block 220 is stored in RAM on at least one 
node in the cluster, ensuring that fingerprint searching occurs very rapidly. It is noted that 
some computational nodes corresponding to the other functional blocks of signal 
identification block 1 10, such as landmarking 210, fingerprinting 215, and alignment 
scan 225 may not require as much bulk RAM as the nodes supporting song database 220. 

20 The number of computational nodes assigned to each module may thus be scaled 

according to need so that no single module becomes a bottleneck. The computational 
network is thus, advantageously, highly parallelizable and can additionally process 
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multiple simultaneous signal identification queries, where such queries are distributed 
among available computational resources. 

In an illustrative embodiment of the invention, some of the functional modules 
may be less tightly coupled together than to the other modules. For example, the 
5 landmarking and fingerprinting functions of blocks 2 1 0 and 2 1 5 of FIG 2 may reside in a 
physically separate location from the rest of the computational devices shown. One such 
example of physical separation would be realized if the landmarking and fingerprinting 
functional blocks 210 and 212, respectively, are more tightly associated with the signal 
capture block 105. In such arrangement, the landmarking and fingerprinting functions 
10 described above are realized and incorporated as additional hardware or software 

embedded in a mobile telephone, WAP browser, or other remote terminal, such as the 
client-end of an audio search engine. In an internet-based audio search service, such as a 
;U content identification service, the landmarking and fingerprinting functions may be 

□ incorporated into the browser application as a linked set of software instructions, or as a 

15 software plug-in module, such as a Microsoft DLL. In such embodiments employing 
loosely coupled functional modules, the combined signal 

capturing/landmarking/fingerprinting functional blocks shown in FIG 2 form the client 
end of the service and send a feature-extracted summary of the captured signal, 
comprised of sample landmark/fingerprint pairs, to the server end (e.g., song database 
20 220 in FIG 2). Sending this feature-extracted summary to the server instead of the raw 
captured audio is advantageous since the amount of data is greatly reduced, often by a 
factor of 100 or more. Such feature-extracted summary could be sent in real-time over a 
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low-bandwidth side channel along with, or instead of, an audio stream transmitted to the 



server. 



FIG 3 shows details of the identification reporting and transactional aspects of the 
presenNnvention embodied in reporting and transaction block 115 (FIG 1) and illustrates 
5 the highly imbgrated response mechanism to the user (i.e., a machine or person) who 
requested the signaHdqitification. As indicated in FIG 3, identification-reporting control 
block 3 10 of reporting and transaction block 115 receives the identification information 
(i.e., song ID) from signal identificatioiH^ock 1 10 via line 113. Identification and 
reporting control block 310 also receives user dkt^, illustratively user options and user 

10 information signals (collectively indicated as lines 1 18 nv^IG 3) from user identification 
block 120 via user database 312 and line 1 17 as indicated. 
£^0**\^ database 3 12 is optionally utilized to store detailed user information and may 
facilitate the\;er billing function as described above. Functional block 314 is interposed 
between user datable 3 1 2 and identification reporting control block 3 1 0, as shown in 

1 5 FIG. 3, to implement data\rodates to the user database (e.g., the user account) on line 316 
with such detailed information aSva function of the input signals 113, 117, and 1 18, to 
identification reporting control block JKhind under the control of identification reporting 
control block 310 via control line 302. Thus, u^r database block 312 operates in a 
read/write manner. It will be appreciated that user datkktase block 312 may be particularly 

20 beneficial in some applications. Any time a user captures a sifeqal for identification, the 
service can leverage existing, and capture new, data and statistical nH^s. First, the service 
can log the signal identification for the user account, and every subsequenibqntinued 
interaction with the service. Second, the service can use existing data about the usbr 
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9 



Spunt to enhance the experience. This will create a highly personalized experience, 
whether ifbe^eusipm user account settings and/or preferences, a personalized user 
website, or targeted advertising. 

Referring again to FIG 3, identification reporting control block 3 10 is 



operatibqally coupled via lines 304 and 306 to real-time reporting block 320 and offline 
reporting blobl^330. The user interaction with the arrangement of the present invention 
may be in real-tim^r delayed. Real-time reporting provides an instant response about 
the identified signal to tftfcyiiser. This real-time response may be in the form of data or 
voice. Voice annotation mean^hat the user learns about the identified signal by listening 
10 to a voice report. After receiving tnb\voice report from voice annotation block 324, the 
user may be provided with additional options for further interaction, such as playback of 
the signal captured and purchase opportunities^ the content identified (i.e., the user may 
place an order to purchase the song or album thatWs just identified). This interaction is 
characterized by being voice prompted, in that interactive voice response user interface 
1 5 block 322, functionally coupled to identification reporting C<mtrol block 3 1 0 through real- 
time reporting block 320 reads out the alternatives, asking the i^er to respond. The user 
provides such responses to the voice prompts provided by the servicMirough keypad 
input or by voice (where in such instances, voice recognition methodologies are 
employed to translate the user's voice responses into usable system inputs). Functional 
20 blocks 324, 326, and 328 in FIG. 3 illustrate several additional options, voice anhotation, 
song excerpt playback, and purchase options, respectively, that may be offered to the^iser 
in accordance with the invention. The purchase options, as noted above, provide a 
solution to the problem of music fans who attempt to purchase a particular song recording 
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after hearingTt^bdng played or performed (for example, from a radio broadcast), but 
cannot recall the name o^th^song or artist. In accordance with this feature of the 
invention, the user may immediately^pujchase the desired recording via real-time 
purchase options block 328 immediately after recehna^the voice report from voice 
5 annotation block 324. 

Alternative options are also contemplated in accordance with the invention: the 
user may hear a clip of the identified recorded song (as compared to the playback of the 
captured signal described above); the user may hear more information about the song, 
artist, or album; the user may hear clips from other songs on the same album or from the 
1 0 same artist as the identified song; or, the customer may choose to interact with customer 
service personnel. Song database 360 is operationally coupled to identification reporting 
control 310 via line 362 to implement such contemplated options. Song database 360 may 
include songs as well as related or other data. 

Djjta responses are used to transmit information about the identified signal back to 
such as a WAP browser interface on mobile telephones, or 
other appropriate protocols over titoSs^In FIG 3, the WAP browser interface block 340 is 
operationally coupled on line 338 to real-timerfcflQrting block 320. Thus, the user has the 
option to interact further with arrangement 100 (FIG 1) byllsing such an interface in 
accordance with the principles of the invention. This particular interacti^b^tween user 
20 and service is characterized by being data prompted and does not need to rely upoir^mQe. 

14eweyer l _a^cor^ of voice and data is also contemplated as falling within 
the scope of the present invention^sho^i in FIG 3, where such a combination creates a 
beneficial and seamless experience for the user. The technoI^Er^latform in accordance 



1 5 the user through an 
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witirBielnvention advantageously integrates both voice and data (respectively through 
IVR and web-based operations, for example) so that the user's experience with a service 
utilizing the principles of the inventionTs^Consistent and seamless across all interfaces 
(including those identified in operational blocks 322, 340, an3^50r«tgractive voice 
response interface, WAP browser interface, and internet browser interface, respectively^ 
V\^CX Delayed reporting takes places at a point in time when the user has disconnected 
from the sfei^ice, i.e., after the step of signal capture shown in signal capture block 105 
(FIG 1). Delayed reporting (or "offline" reporting), whether voice or data, is 
accomplished by sending mfonnation about the identified signal to the user via an 
10 Internet browser, email message, S^message or other communication methodologies. 
This feature of the present invention is showain FIG 3 with interface browser interface 
block 350, SMS text messaging block 360 and em^Rinformation block 370 being 
operationally coupled to offline reporting block 330 on lifres 332, 334, and 336, 
respectively. A combination of the three modes of offline reportihgis possible and may 
1 5 be preferred in some applications of the invention. 

^^C^ -Qel^ed reporting further may include the option for further interaction with a 
^ service whiclrutU^^ principles of the present invention, such as playback, purchase 
opportunities, and the like. Tlfe^btocks operationally coupled and depicted to the right of 
real-time reporting block 320 and offline^epOTting block 330 in FIG 3 thus represent data 
20 and/or signal outputs from reporting and transaction&teQk 1 1 5 to users or other 

constituencies. More particularly, with respect to real-time reporting, interactive voice 
response user interface block 322 provides output data from voice annota&sq^lock 324, 
song excerpt playback block 326, and real-time purchase options block 328 back to*^ 
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user (in this illustrative example of the invention via the return channel of the duplex call 
on the user's mobile telephone) as shown on line 372 of FIG 3. Similarly, WAP browser 
interface block 340 and online information browsing options block 342 provide 
interactive data output to the user on line 374. 
5 ^^^"Wfth^ to offline reporting, internet browser interface block 350 and online 
^purchase options block 352^roVtde^utput on line 376 while SMS text messaging block 
360 and email information block 370 provideoulput^ata, to the mobile telephone user in 
this illustrative example of the invention, via lines 378 and 380^510^. Output from 
reporting and transaction block 1 15 is also directed to statistical data logger liO^EJG 1). 



10 



Ssj^ ^t^with real-time purchase option block 328 in FIG 3, WAP browser interface 
J 340 and internet Wowser interface 350 are operationally coupled to online purchase 



options block 352 on lines 3^Hnd 351, respectively. Online purchase options block 352 
may implement the same type of ftmctiohalities and alternatives options discussed when 
describing real-time purchase options block 328 ab^€^Similarly, online information 
1 5 browsing options block 342 is cross coupled to receive input frbm4nternet browser 
interface block 350 and WAP browser interface block 340 on lines 343 anc 
respectively. 

A combination of real-time and offline reporting may be advantageously 
employed in some applications of the invention. In such combinations, the user 
20 distributes interactions with the service or particular aspects of interactions using both the 
real-time and offline interfaces over a period of time or at the same time. For example, a 
user may place an order for an identified song over the phone (the real-time interface), 
and then provide payment and delivery information for that order later through the 
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service's website (the offline interface). Likewise, a user may wish to interact with the 
IVR in real-time over the phone while simultaneously interacting with the service to 
arrange for SMS message about a particular song to be sent to a friend's mobile phone. 
Of course, other beneficial combinations may be readily arranged using the principles of 
the present invention. 

(Staline information browsing options block 342 may be used in certain 
applications oMie invention to implement a number of desirable functionalities and 
features. For examptfe^a user using a WAP or Internet browser could access a service 
provider's website which utilizes the features provided by online information browsing 
options block 342 in FIG 3 in accohlance with the invention, to recommend songs to 
friends, chat with other service users, playgames (e.g., a game where users try to identify 
obscure song tracks sampled by other users), ancN}ther activities that are facilitated by 
internet's large reach. In addition, information browsin^ptions block 342 may be used 
to implement the delivery of promotional materials (such as chj3S from albums) and 
special event tickets or merchandise, or manage archived data selected by that user such 
as sample and "wish" lists. Information browsing options block 342 may al^be used to 
implement an interaction with the user to manage or search for other information,^ 

The present invention also includes features implemented using information- 
browsing options block 342 to allow users to set up alerts for news releases and concert 
announcements. Users could also interact with a service utilizing this feature of the 
invention to send information to friends (via SMS or email, for example) on the types of 
music they like, or send an audio clip directly to the friend's mobile telephone. A wide 
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range of services and features, in fact, may be readily implemented using the data 
response features of the present invention. 

While the present invention contemplates that reporting, both real-time and 
delayed, is primarily targeted to the user who requested identification, it may be desirable 
5 in some application of the inventions to include delayed reporting to other constituencies. 
This reporting occurs via a service utilizing the principles of the present invention and not 
directly from the user. For example, a person who identifies a song, can request the 
service to send an SMS or email message about the identification to other people as well. 
Turning again to the present illustrative example of the invention where a user is 

10 located in a high noise environment such as an automobile, interactive voice response 
user interface block 322, in accordance with the invention, is utilized to minimize human 
intervention. First time users may interact with a human operator, in this illustrative 
example, using a mobile telephone, to register, for example, their email address, user 
profile, and other details relating to unregistered users who may wish to make purchases. 

1 5 Communications may also be made with the user through email and website 

interactions. After the initial registration, a telephone interface enables users to dial a 
particular number to initiate a call to the service's IVR. The IVR identifies the caller 
through the user's mobile telephone number in accordance with operations of functional 
block 120 (FIG. 1). The user can then record a music sample, which in this example is 

20 being played on the user's car radio, which is captured and recognized (i.e., identified) as 
described above. 

£^V^ "" With the cx 66ptioni)fjhe^ registration, calls with not require 

human intervention. The user may chose to receivTrvoiee-repor^ebyed back on the 
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"mobile which^rovides^e^esi^esong identification from voice annotation block 324 
(FIG 3), or optionally, the user may receive^rem^ilwith the name of the track, artist and 
album title, and links to the service provider's website which facHftatesJheuser accessing 
a range of services as described above. 
5 In another illustrative example of the invention, a user is listening to radio or 

television programming while working at a computer workstation. After an initial 
registration process similar to that described above (which may be accomplished over the 
telephone as with the previous example, or via online registration), the user may record a 
music sample using the computer's microphone and then access the service provider's 
10 facilities via website or dial up to allow the sample to be uploaded as a sound file, 
captured, and identified in accordance with the present invention. In this example, 
identification of the user may be effected through the user's IP address. Other features 
and functions in this example are similar in scope and operation to those described in the 
example above. 

1 5 Although various embodiments are specifically illustrated and described herein, it 

will be appreciated that modifications and variations of the invention are covered by the 
above teachings and within the purview of the appended claims without departing from 
the spirit and intended scope of the invention. For example, while several of the 
embodiments depict the use of specific communication techniques and protocols between 

20 various embodiments, any communication technique will suffice to transfer information 
between the two devices. Moreover, while some of the embodiments describe specific 
recording formats, any data and information format for transferring data to the user may 
be employed by the invention described herein. Furthermore, these examples should not 
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be interpreted to limit the modifications and variations of the invention covered by the 
claims but are merely illustrative of possible variations. 



-40- 



